Reconstruction sequences and equipartition measures: An examination of the asymptotic equipartition property

We consider a stationary source emitting letters from a finite al­ phabet A. The source is described by a stationary probability measure a on the space 0 := AlN of sequences of letters. Denote by On the set of words of length n and by an the probability measure induced on On by a. \Ve consider sequences {rneOn : n E IN} having special properties. Call {rneOn: n E IN} a sup­ porting sequence for a if limn an[rn] = 1. It is well-known that the exponential growth-rate of a supporting sequence is bounded below by hSh(a), the Shannon entropy of the source a. For efficient simulation, we require r n to be as large as possible, subject to the condition that the measure an is approximated by the equipartition measure f3~n, the probability measure on On which gives equal weight to the words in r n and zero weight to words outside it. We say that a sequence {rneOn : n E IN} is a reconstruction sequence for a if each r n is in­ variant under cyclic permutations and limn JJ~n = am for each m E IN. We prove that the exponential growth-rate of a reconstruction sequence is bounded above by hsh(a). We use a large-deviation property of the cyclic empirical measure to give a constructive proof of an existence theorem: if a is a stationary source, then there exists a reconstruction sequence for a having maximal exponential growth-rate; if a is ergodic, then the reconstruction sequence may be chosen so as to be supporting for a. We prove also a characterization of ergodic measures which appears to be new.